Video analytics platform for real-time monitoring and assessment of airplane safety processes

ABSTRACT

A video analytics system for real-time monitoring and assessment of airplane ramp safety processes comprises rules relating to airplane ramp safety processes, portable video capture devices located at different predetermined positions with respect to an airplane at an airport ramp which capture video and data related to movements and positioning of objects and services being provided relative to the aircraft at the airport ramp and transmit captured video footage and data via a wireless network to a server which performs video analytics on the captured video and data to generate video analytics data, compare the video analytics data to the rules relating to the airplane safety processes, to determine any safety conditions, store the results of the comparison and output a real-time notification if a safety condition is determined.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. Patent Application Ser. No.17/318,043, filed May 12, 2021, entitled “VIDEO ANALYTICS PLATFORM FORREAL-TIME MONITORING AND ASSESSMENT OF AIRPLANE SAFETY PROCESSES”, whichclaims the benefit under 35 U.S.C. § 119(e) of U.S. ProvisionalApplication Ser. No. 63/119,327, filed Nov. 30, 2020, entitled “VIDEOANALYTICS PLATFORM FOR REAL-TIME MONITORING AND ASSESSMENT OF AIRPLANESAFETY PROCESSES”, which are hereby incorporated by reference in theirentirety.

TECHNICAL FIELD

The disclosed technology relates to a video analytics platform forreal-time monitoring and assessment of airplane safety processes.

BACKGROUND

Fixed video surveillance technology installed throughout airportoperations for the purposes of securing restricted areas is known. Thevideo may be used to provide notifications when a person breaches asecure area. For example, a video camera may be directed at a door(e.g., an airport gate, security check, etc.) and a securityadministrator may watch the live stream of video from a secure location.When a person enters or exits through the door, the securityadministrator may identify the activity by watching the live stream ofvideo. Typically, the video camera may be mounted at a fixed located(e.g., directed at the door and/or airport gate).

Known video systems have a number of limitations and are typically forsecurity purposes, but they are not typically used for monitoring safetyprocedures.

For example, a video security camera may be concerned with a person thatis adjacent to an airplane. A security system may be more concerned withwhether an authorized person in proximity to an airplane is wearing acolored safety vest.

For these and other reasons, existing airport video security systems arenot used and/or effective for real-time monitoring and assessment ofairplane safety processes.

Airports and airlines have manual procedures to track safety processes.However, these are typically manual processes, and the relevantinformation is not collected and/or processed in real-time to preventsafety issues in real-time.

Other limitations and drawbacks exist with known systems.

BRIEF SUMMARY OF EMBODIMENTS

Various methods, systems, and computer-readable media are disclosedherein. In some embodiments, the invention relates to a system andmethod for monitoring airport ramp safety processes and procedures bycapturing video of the ramp area, analyzing the captured video, andcomparing the content to stored documented safety processes andprocedures to improve safety. The analysis may also help mitigate eventsthat could lead to injury or damage to equipment by providingnotifications in real-time or offline.

The system may include a customized, weather resistant, portable videocapture device for capturing data for real-time or off-lineobservations. Two or more video capture devices may be selectivelypositioned relative to the location of the aircraft based on the type ofaircraft being monitored and other factors (e.g., the services beingprovided to an aircraft) . These selectively positioned video capturedevices would be placed in a manner to provide a range of visibility tocover the entire safety diamond area of a parked aircraft. In someexamples, the video capture devices may be stationary or moveable forcapturing data.

The video capture device may collect video footage and data related tomovements and positioning of objects and services being providedrelative to an aircraft. The video capture device may include a Wi-Fi orcellular network interface for the uploading of data in real-time and orin batch mode.

The video and data may be wirelessly transmitted or otherwise conveyedfrom the video capture device to a server. The server may include areal-time transaction engine (RTE). The RTE may perform analytics on thevideo, store information obtained from the video, compare the videoanalytics data to stored rules and content relating to stored, safetyprocesses and procedures. The results of the comparison can be stored.If certain conditions are detected, the RTE may output real-timenotifications and/or provide alerts in real-time to address safetyconcerns.

The system may include a server that communicates with the portablevideo capture devices. The system may store and process data and video.The system may include a user interface that provides users withanalytical data and historical video used to improve the operation andsafety practice.

The system may include an artificial intelligence/machine learningengine to process the captured data.

Other features and aspects of the disclosed technology will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, which illustrate, by way of example, thefeatures in accordance with embodiments of the disclosed technology. Thesummary is not intended to limit the scope of any inventions describedherein, which are defined solely by the claims attached hereto.

BRIEF DESCRIPTION OF THE DRAWINGS

The technology disclosed herein, in accordance with one or more variousembodiments, is described in detail with reference to the followingfigures. The drawings are provided for purposes of illustration only andmerely depict typical or example embodiments of the disclosedtechnology. These drawings are provided to facilitate the reader'sunderstanding of the disclosed technology and shall not be consideredlimiting of the breadth, scope, or applicability thereof. It should benoted that for clarity and ease of illustration these drawings are notnecessarily made to scale.

FIG. 1 illustrates a real-time transaction engine, video camera unit,and client device, in accordance with some embodiment discussed herein.

FIG. 2 illustrates camera placement around an operational safety zone(OSZ) or Safety Diamond, in accordance with some embodiment discussedherein.

FIG. 3 illustrates camera placement around an operational safety zone(OSZ) or Safety Diamond, in accordance with some embodiment discussedherein.

FIG. 4 is an illustrative machine learning (ML) model and data pipeline,in accordance with some embodiment discussed herein.

FIG. 5 illustrates a graphical user interface providing an executivedashboard, in accordance with some embodiment discussed herein.

FIG. 6 illustrates a graphical user interface providing a monitoringreport dashboard, in accordance with some embodiment discussed herein.

FIG. 7 illustrates a graphical user interface providing a monitoringdetails dashboard, in accordance with some embodiment discussed herein.

FIG. 8 illustrates a graphical user interface for providing a searchfunction within an executive dashboard, in accordance with someembodiment discussed herein.

FIG. 9 illustrates a graphical user interface for providing complianceaverages, in accordance with some embodiment discussed herein.

FIG. 10 illustrates a graphical user interface for providing a totalaverage errors per turn panel, in accordance with some embodimentdiscussed herein.

FIG. 11 illustrates a graphical user interface for providing acompliance percentage panel, in accordance with some embodimentdiscussed herein.

FIG. 12 illustrates a graphical user interface for providing a totalerror panel, in accordance with some embodiment discussed herein.

FIG. 13 illustrates a graphical user interface for providing a searchfilters panel, in accordance with some embodiment discussed herein.

FIG. 14 illustrates a graphical user interface for providing amonitoring details panel, in accordance with some embodiment discussedherein.

FIG. 15 illustrates a graphical user interface for providing amonitoring report, in accordance with some embodiment discussed herein.

FIG. 15A illustrates a graphical user interface for providing anobservation improvement report, in accordance with some embodimentdiscussed herein.

FIG. 15B illustrates a graphical user interface for providing areal-time alert dashboard, in accordance with some embodiment discussedherein.

FIG. 16 is an example computing component that may be used to implementvarious features of embodiments described in the present disclosure.

The figures are not intended to be exhaustive or to limit the inventionto the precise form disclosed. It should be understood that theinvention can be practiced with modification and alteration, and thatthe disclosed technology be limited only by the claims and theequivalents thereof.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Various components of a video analytics platform are described herein.The video analytics platform may be configured to provide real-timemonitoring and assessment of airplane safety processes.

Video Analytics Platform

FIG. 1 may comprise real-time transaction engine 110 that communicatesvia network 132 to both video camera unit 150 and client device 160.Each of real-time transaction engine 110, video camera unit 150, andclient device 160 may comprise processors, memory, antennas, and othercomponents of computing devices to enable these computing devices tocommunicate via network 130 and execute machine-readable instructions.

Real-Time Transaction Engine

Real-time transaction engine 110 comprises video processing circuit 112,machine learning circuit 114, and reporting circuit 116. Additionalcircuits and components may be included without diverting from the scopeof the disclosure.

Real-time transaction engine 110 is configured to receive video, audio,and other media from video camera unit 150 via network 130. The mediadata may comprise footage captured from around an airport facilityregarding ramp operations, safety checks, or other information used tovalidate compliance with safety rules, policies, and procedures. Thevideo data may be wirelessly transmitted or otherwise conveyed fromvideo camera unit 150 to real-time transaction engine 110.

Real-time transaction engine 110 is configured to provide videostreaming (e.g., WebRTC) and related capabilities. The capabilities mayinclude transcoding inbound and outbound videos from/into various videoand audio codecs and formats, recording and storing videos or individualframes on the server, transcoding and compressing videos, ingesting andprocessing multiple video streams concurrently, applying ML and non-MLfilters to video streams and individual frames, providing SFU (SelectiveForwarding Unit) and MCU (Multipoint Conferencing Unit) functionalitiesenabling videoconferencing and video streams mixing, providing TURN andSTUN servers functionalities allowing better routing of WebRTCconnections and NAT traversal, supporting combined media pipelinesallowing insertion of Computer Vision, statistical and other filters andmodules into media processing pipeline, cross-platform WebRTC videostreaming to iOS, Android, React Native, Web and desktop applications,the capability of applying augmented reality visualizations, masks andeffects in 2D and 3D to real-time video streams, the capabilities ofmixing and blending video streams as well as video calling, groupcommunication and broadcasting.

Video processing circuit 112 is configured to perform analytics on thevideo. For example, the analytics may identify metadata of the video,including location, originating video camera unit 150, date or timestamp, and other information. The information may be stored with imagesdata store 120.

Video processing circuit 112 is configured to recognize one or moreobjects in the video. For example, video processing circuit 112 maycompare known images stored with images 120 with received images fromvideo camera unit 150. Video processing circuit 112 may tag the one ormore objects with an object identifier based on the comparison betweenthe known images and the identified image in the received video. Theobject identifier, timestamp, video portion, metadata, or otherinformation (e.g., generated by optical character recognition (OCR),etc.) may be stored in images data store 120.

Video processing circuit 112 is configured to augment or add to themedia data. For example, video processing circuit 112 is configured toprovide video, image, and web interface augmentation capabilities inrelation to Artificial Intelligence, Computer Vision detection, andrelated user interfaces such as preconfigured and pre-developedcapabilities to visually highlight detected objects and scenes, tohighlight movements of most importance, to overlay characteristics ofdetected objects and events.

Video processing circuit 112 may compare the object identifier,timestamp, video portion, metadata, or other information from imagesdata store 120 with rules from rules data store 122. Rules data store122 may comprise various content relating to safety processes andprocedures.

Video processing circuit 112 may compare the object identifier generatedfrom the video analytics to stored rules from rules data store 122. Asan illustrative example, when a vest is required at a particularlocation, video processing circuit 112 may determine that the objectidentifier from the tagged media identifies a vest in the video. Videoprocessing circuit 112 may confirm that the stored rule is satisfiedbased on the object identifier found in the media. The results of thecomparison may be stored in results data store 124. Additionalinformation and illustrative rules stored with rules data store 122 areprovided herein with the subsection labeled “illustrative examples inrules data store.”

Machine learning circuit 114 is configured to receive media input,provide the media input to one or more trained machine learning (ML)models, and generate output in view of an objective functioncorresponding with each trained ML model. Additional informationregarding the machine learning process is provided with FIG. 4.

Reporting circuit 116 is configured to generate one or more reports,alerts, notifications, or other information based on the comparisons ordeterminations described herein. The reports and alerts may be generatedin real-time to correspond with the real-time streaming video from videocamera unit 150. If certain conditions are detected, reporting circuit116 may output real-time notifications and/or provide alerts inreal-time to address safety concerns.

Reporting circuit 116 is configured to transmit generated reports on apredetermined basis (e.g., daily, etc.) to designated client devices160. In some examples, the reports may be transmitted in accordance withan event (e.g., Daily Safety Briefings, Monthly Station Safety Meetings,one-on-one briefings, Safety Huddles, etc.).

Reporting circuit 116 is configured to provide other data to a userinterface. For example, reporting circuit 116 may provide a smarttimeline, dashboard, and/or video player. The other capabilities mayinclude viewing the video stream in real-time in a web application,mobile application, or desktop application video player interfaceallowing to stream one or more videos simultaneously; playing, stopping,pausing, rewinding, fast-forwarding the videos; switching between cameraviews and/or open pre-recorded videos; capability to highlight importantdetections with color coding and other interface elements in the videoplayer timeline; capability to skip/fast-forward intervals of noimportance as determined by the system or user settings; capability tosearch through stored videos and highlight in search results and in thevideo player timeline the occurrences of the detection or event user issearching for; capability to display and store the internal consoleshowing lower level detections, logging and other system information aspart of user interface; dashboard capability allowing to displaymultiple video streams at the time, overlay, and augment videos, zoom inand zoom out videos, including capabilities of automated triage anddetection of most important detections or events among displayed andnon-displayed video streams and ability to automatically zoom in andalert human operator of the displayed or non-displayed video wherehighest ranking detections or events are happening.

Reporting circuit 116 is configured to provide smart feedback. Thecapabilities may include providing an overlay of the user interface toprovide end users with an easy way to communicate false positive andfalse negative detections to the system; automated ML and non-ML basedmethods to infer false positive and false negative detections from userinteractions or inactivity; automated re-training and re-learning loopsystem automatically applying false negative and false positivedetections to re-learning or transfer learning of ML models.

Reporting circuit 116 is configured to transmit electronic messages vianetwork 130. The messaging system may be based on various communicationprotocols (e.g., Extensible Messaging and Presence Protocol (XMPP)instant messaging protocol, Ejabberd® server, accompanying custom-madeservices and APIs, etc.). The messaging may allow end users andautomated systems (such as chat bots, ML and non-ML detects, alertingsystem, etc.) to carry out text-based communication between each othervia existing interfaces of the system and/or integrated into 3rd partymobile, web, and desktop applications.

Reporting circuit 116 is configured to transmit alerts or statistics.The capabilities may include push notifications, e-mail, user interfaceaudio and visual alerts providing the designated contacts with real-timeand summary alerts regarding detections, conclusions, and otherinformation from the system.

Reporting circuit 116 is configured to generate a compliance score. Thecompliance score corresponds with a number of rules that are satisfiedbased on a comparison between the media data and one or more rules fromrules data store 122. The compliance score may be of value (e.g., 100)and each image that identifies that a rule is not satisfied may reducethe compliance score. In some examples, the goal may be to achieve 100%compliance or satisfaction of each of the rules for the location fromrules data store 122.

The compliance score may be compared with a score threshold. Correctiveaction may be identified based on the comparison of the score to thescore threshold. In some examples, different score thresholds maycorrespond with different actions.

Reporting circuit 116 is configured to generate one or more reportsbased on different stages, timestamps, locations, and the like. Anillustrative example is shown herein.

Average Action Plan Stages Goal Compliance Required Arrival 100 100 NPost-Arrival 100 93.1 N Post-Departure 100 56.52 Y Pre-Arrival 100 74.03Y Pre-Departure Preparations 100 79.82 Y Upload 100 57.14 Y

Reporting circuit 116 is configured to generate a corrective actionplan. When developing the corrective action plan, the following templatemay be used to drive consistency and to ensure the plan meets variousrules stored in rules data store 122. The action plan may be designed toaddress each individual element that has been identified (e.g., mediadata, rule, etc.), describe what will be implemented or put into placeto improve compliance level(s), establish a timeline for when the planobjectives are expected to be achieved, indicate the person(s)responsible for implementation, and follow up and verification ofobjective effectiveness.

Reporting circuit 116 is configured to generate and update a dashboardto display information in real-time. Historical media may also beaccessible and provided via the dashboard. The dashboard may beaccessible by client device 160 and provided for display at userinterface 162. Illustrative examples of reports and reporting data areprovided with FIGS. 5-15.

Authentication circuit 118 is configured to authenticate client device160 and/or user with access to real-time transaction engine 110. Variousreports and/or data may be limited based on a user type associated withthe user accessing the system. A username, user type, or otherinformation may be stored with a user profile in user profile data store126.

In some examples, real-time transaction engine 110 may be configured tomonitor and analyze the system itself. Monitoring and analytics maycorrespond with API, server, and infrastructure parameters allowing tomonitor and visualize overall system performance such as uptime andlatency of APIs, throughput of media processing and machine learningpipelines, activity, and availability of sources (e.g., video cameraunit 150, other video, audio, and data streams, etc.), load diagrams forCPU and GPU servers, data storages, queues etc., including uptimewidget, detailed monitoring dashboard, master dashboard and alerts viae-mail, SMS, phone calls, chat messages and push notifications;underlying architecture, storage systems and server-side logic (e.g.,MongoDB®, Apache® Cassandra®, MySOL®, Redis®, IPFS®, HDFS®, Apache®Kafka®and Zookeeper®, Kurento®, Janus®, Ejabberd®, Apache® Spark®,Apache® Flink®, Zabbix®, Grafana®, Prometheus®), APIs, backend(server-side) code, and documentation. This includes logic allowing tocreate and manipulate user accounts, etc.

Client device 160 may access one or more reports generated by reportingcircuit 116 and/or reporting data from results data store 124 vianetwork 130. The reporting data may be displayed for presentation at thereporting dashboard by user interface 162. In some examples, clientdevice 160 may receive push notifications from reporting circuit 116 vianetwork 130.

Custom-Designed Video Camera Units

Video camera unit 150 may comprise an off-the-shelf or custom designedvideo camera assembly. The video camera assembly may include a videocapture (e.g. a video camera) encased in a ruggedized housing. Theassembly may further include a transmitter (e.g., a wireless, cellularor other transmitter) to receive the video and transmit it in real-timeto one or more designated locations (e.g., a server associated with anairport, a central server remote from the airport and/or otherlocations). The assembly may also include a storage device to store thevideo captured by the video camera.

In some examples, video camera unit 150 may comply with rules accordingto Ingress Protection (IP) numbers. For example, video camera unit 150may comply with IP67 (e.g., totally protected against dust and protectedagainst the effects of temporary immersion between 15 cm and 1 m, orduration of test 30 minutes) and IP68 (e.g., totally protected againstdust and protected against long periods of immersion under pressure).Video camera unit 150 may be environmentally tested.

Video camera unit 150 may be encased in a durable, rugged utility casefor stability and protection from environmental factors. The case may beequipped with a handle and wheels used to manually roll the unit to oneor more locations around the aircraft facility.

Video camera unit 150 may comprise one or more arrows on the case tofacilitate proper orientation and alignment. For example, the case maybe directed at an angle and location such that the arrow may be pointedtoward the aircraft. The position of the arrow may align video cameraunit 150 so that the camera lens may capture media corresponding withthe aircraft from the particular location and perspective.

Video camera unit 150 may generate a real-time video stream. Forexample, the camera incorporated with video camera unit 150 may captureimages, audio, or other media and store the data locally with videocamera unit 150 or transmit it to real-time transaction engine 110 vianetwork 130. Video cam unit 150 may comprise an antenna (e.g., forwireless communications) and/or physical port (e.g., for wiredcommunications).

Video camera unit 150 may comprise a Wi-Fi or cellular network interfacefor communicating with network 130 and/or uploading the data.

Video camera unit 150 may comprise a camera lens. For example, thecamera lens may comprise a wide angle lens for capturing images adjacentto camera unit 150.

Video camera unit 150 may be static or stationary. In some examples,video camera unit 150 may be uniquely placed based on aircraft typeand/or other factors. This may comprise internal placements of aparticular aircraft, including a first video camera unit 150 on a firstfloor of an aircraft and a second video camera unit 150 on a secondfloor of an aircraft when the aircraft corresponds with more than onefloor.

Video camera unit 150 may be placed externally to an aircraft. Videocamera unit 150 may be positioned each morning or prior to the firstoperation at designated gates. The units may be placed outside theoperational safety zone (OSZ) and are left in position until end of dayand/or end of operations at a designated gate. The units may then beretrieved and returned to the docking station for recharge and videodata upload. Two cameras are deployed at each designated gate; one atthe wingtip, and one at the nose of the aircraft, well behind and/or tothe driver's side of the pushback unit.

Deployment And Placement Guidance

Video camera unit 150 may be movable and manually rolled to a designatedlocation. The location to place the one or more video camera units 150may be determined based on the type of aircraft being monitored andother factors, including the services being provided to the aircraft(e.g., filling gasoline to the aircraft, adding food, safety checks,etc.).

In some examples, video camera unit 150 may be manually moved to alignwith a marking on the ground, which may be placed at a differentlocation for each airport and gate. The markings may help align theaircraft to the gate in order to properly service the aircraft while theaircraft is stationary at the gate. In an illustrative example, a Boeing737 may align with Marking 10 and Marking 12, while a Cessna 172 mayalign with Marking 30 and 31. Video camera unit 150 may be manuallyrolled to a location corresponding with each marking that is utilized bythe aircraft (e.g., 10 feet from Marking 10 and Marking 12, 5 feet aboveMarking 30, etc.) in order to capture and transmit the video, audio, andother media. In another example, video camera unit 150 may be manuallymoved to align with a wingtip of the aircraft and a nose of the aircraftin accordance with rules from rules data store 122, unrelated to themarkings.

In some examples, video camera unit 150 may be stationary and affixed tocorrespond with the markings used by the aircraft. For example, whenforty markings are incorporated with a gate at an airport, forty camerasmay be aligned with the markings. The markings that the aircraft uses toalign with the gate may also activate the camera corresponding with themarking. As described with the example above, the two video camera units150 associated with Marking 10 and Marking 12 may be activated when aBoeing 737 arrives at the gate, which also aligns with Marking 10 andMarking 12. In another example, the two video camera units 150associated with Marking 30 and 31 may be activated when a Cessna 172arrives at the gate, which also aligns with Marking 30 and 31. In otherexamples, each video camera unit 150 may be continually active andstationary to capture video of the aircraft, without the activationprocess and without diverting from the essence of the disclosure.

In some examples, servicing of the aircraft may focus on the port orleft side of the aircraft as illustrated on regional aircraft such asthe CRJ700. Two or more video camera units 150 may be assigned to asingle aircraft. Thus, video camera unit 150 may be positioned off theleft wing outside the operational safety zone (OSZ) or Safety Diamond inthe general vicinity as shown in the FIG. 2. Illustrative locations fora plurality of video camera units 310 (illustrated as video camera unit310A, 310B, 310C, 310D, 310E, 310F) and particular aircrafts 320(illustrated as aircraft 320A, 320B, 320C) are provided.

In some instances, video camera unit 150 may be placed where it wouldhave a vantage point covering the entire operational safety zone (OSZ)or Safety Diamond in order to see all of the operational activities thatoccur while servicing the aircraft, as illustrated in FIG. 3. Forexample, the camera placement may be on the opposite wing side if theobserved aircrafts cargo compartments exist on the port side versus thestarboard side.

The markings may be placed in other locations as well, including a ramp,front/back of an aircraft, security door, or other location maycorrespond with a rule from rules data store 122. In some examples, therules (e.g., location of one or more video camera units 150) maycorrespond with an airplane type.

Arrows incorporated with each video camera unit 150 may facilitateproper orientation and alignment. For example, an arrow may be printedon top of the rugged casing. Generally, the arrows will be pointedtowards the aircraft. The lenses are wide angle and capture a lot ofarea. Fuel tenders and catering trucks may block the entire view for anextended period of time depending on service requirements. Thus, becognizant of the camera position when such ground equipment is used.

Illustrative Examples In Rules Data Store

Rules data store 122 may comprise a plurality of rules.

An illustrative rule may comprise a 3-stop brake check. The rule mayassume the vehicle is clearly visible at all times. Image data may becompared with images corresponding with this rule to determinesatisfaction of the rule. Satisfaction of the rule may confirm that (1)the vehicle has carried out three stops, (2) the approximate distancebetween stops, and (3) whether the distance falls under complianceparameters (around 50% distance between each stop, etc.).

An illustrative rule may comprise handrails on Ground Service Equipment(GSE) being used. The camera setup may be adjusted to determinesatisfaction of this rule (e.g., the camera line of sight may confirmthat the image data clearly distinguishes between person gripping thehandrail and person simply moving their arm as they are walking besidethe handrail, etc.). Satisfaction of the rule may confirm objectdetection (hand gripping handrail) and pose estimation (detectingperson's skeletal and hand movements and trying to infer whether theyare leaning on or gripping the handrail, detection based on 2 kneespresent, belt loader+working area tracking, etc.).

An illustrative rule may comprise pre-arrival safety huddle conducted athuddle cone. Satisfaction of the rule may confirm detection of 3 or morepersons stopping for 20-60 seconds near to a cone.

An illustrative rule may comprise a lead marshaller and wingmarshallers/walkers in correct position. The camera setup may beadjusted to confirm media data is captured of marshallers, includinglead marshaller, and wing marshallers positions. The camera setup may beadjusted to detect wing marshallers positions that correspond with theaircraft size. The camera setup may be adjusted to detect safety zonemarkings on the ground. The camera setup may be adjusted to determineaircraft pose estimation. In some examples, aircraft and workerscoordinates may be connected with a digital map (e.g., 2-dimensional) orgeolocation device.

An illustrative rule may comprise detection of employees wearing safetyvests secured to their body. Satisfaction of the rule may confirm thevest is “Secured to body” (e.g., zipped, over two shoulders, across theback and front of the body, etc.) or detection of an unsecured vest(e.g., unzipped or unbuttoned vests, etc.).

An illustrative rule may comprise cones placed in proper positions andtimely. The camera setup may be adjusted to detect cones between thecamera and the aircraft, including cones behind aircraft. Satisfactionof the rule may confirm that (1) detect all cones visible in the sceneand (2) demonstrate the approach allowing to detect wing cones, tailcone, and front wheel cone being in position.

An illustrative rule may comprise a detection of a belt loader as anobject and/or detect stops they are making. The rule may detect that abelt loader is making a stop at a stop sign or other required stoplocation. The analysis may utilize a combination of ML and non-MLmethods to detect belt loaders as objects and to detect the stops theyare making. For example, real-time transaction engine 110 may determinewhere the loader is (e.g., geolocation, relative or adjacent objects,etc.) and transfer the coordinates into an array. The analysis maycomprise smoothing the data with one of the filters (e.g., arithmeticmean, running average, Kalman filter), after which the averaged data for5 passes is passed through a function that compares how much thecoordinate has shifted compared to the last pass and after a smalllogical function that checks if the coordinate moved after the lastcheck. This may confirm a new stop.

Machine Learning Pipeline

The system may incorporate one or more machine learning (ML) models. Insome examples, the ML model may be pre-built and/or pre-configuredplatform with processing pipelines and infrastructures consisting oflibraries (e.g., pre-configured TensorFlow, Yolo, PyTorch libraries),weights, models, and software code (e.g., C, C++, Python, Erlang andNode.js).

The models may vary from gate to gate, airport to airport, and may varyby aircraft type. In some examples, the models may vary by gate andairport based on where the video capture units will be placed. For themodels to be trained to a high degree of precision, the ML models mayconsistent placement zones which will not be possible to be identicalacross every gate and airport.

Multiple machine learning models may be implemented. For example, afirst ML model may identify an aircraft and a second ML model mayidentify the services required for the identified aircraft. For example,a first ML model may identify a model of aircraft that is entering theoperational safety zone (OSZ) or Safety Diamond and a second ML modelmay apply the correct camera locations and/or markings for services.

The ML model may correspond with linear or non-linear function. Forexample, the ML model may comprise a supervised learning algorithm thataccepts the one or more input features associated with video data (e.g.,streaming file, etc.) to provide a score.

In some examples, when a nonlinear machine learning model is used, theweightings of fields corresponding with the video data may varyaccording to one or more object identifiers corresponding to the mediadata. This may be illustrated by a presence of an object (e.g., a vest,a safety cone, etc.) at a location corresponding with satisfying asafety rule for that location. In some examples, the weight may bedecided through an iterative training process for each ML model.

In some examples, the ML model may comprise a Deep Learning NeuralNetwork, consisting of more than one layer of processing elementsbetween the input layer and the output later. The ML model may furtherbe a Convolutional Neural Network, in which successive layers ofprocessing elements contain particular hierarchical patterns ofconnections with the previous layer.

In some examples, the ML model may comprise an unsupervised learningmethod, such as k-nearest neighbors, to classify inputs based onobserved similarities among the multivariate distribution densities ofindependent variables in a manner that may correlate with activity thatdoes not correspond with the safety rules or regulations.

Prior to receiving the input features associated with the video data,the ML model may be trained using a training data set of historicalvideo data or standardized segment data for a particular aircraft. Forexample, the training data set may comprise a plurality of images andlocations of an airplane that identify compliance with a safety rule.The correlation of the image to the safety rule may help determine oneor more weights assigned to each of these input features.

In some examples, the ML model may be incorporated with a servicefunction call. For example, real-time transaction engine 110 maytransmit input data to a third party that generates the trained MLmodel. The third-party may provide the input data to the trained MLmodel and transmit output back to real-time transaction engine 110.Real-time transaction engine 110 may incorporate the output as, forexample, correlating media data with a prediction of compliance with asafety rule.

An illustrative ML model and data pipeline is provided with FIG. 4. Insome examples, the ML model and data pipeline is executed by real-timetransaction engine 110 illustrated in FIG. 1.

At block 410, input may be received. Input may be received from varioussources, including real-time video stream data 410A, historicalobservation data 410B, or third-party data 410C.

At block 412, the input may be provided to real-time transaction engine.In some examples, the ML models may be previously trained ML models andthe data may be provided to the ML models in real-time.

At block 414, the input may be stored in a data store. Data from thereal-time transaction engine may be synchronized and/or stored with thedata store.

At block 416, the data may be provided to the ML pipeline. The MLpipeline may comprise online or off-line models. The ML pipeline maycomprise data collection, data transformation, feature engineering,feature selection, model training, tuning and validation, and testing.

At block 418, output from the ML pipeline may be provided to a datascience review. The output may comprise ranking and/or scoring. As anillustrative example, the input may include a video stream of a walkaround a plane with one or more safety issues identified in the videostream. The output may correlate the images found in the video streamwith rules, predefined safety issues, or other data that is feedback forfuture machine learning models. The ML models can use the output fromthe data science review to identify additional safety issues in futurevideos.

At block 420, observation results, reporting, and/or real-timenotifications may be provided. For example, the safety issues may beidentified by the ML model and additional information may be includedwith each safety issue. The safety issues may be ranked and/or scored inorder of importance and the like. The list of safety issues, observationresults, reporting, and/or real-time notifications may be provided to agraphical user interface (GUI). An illustrative example is provided withFIGS. 5, 12, 15, 15A, and 15B.

Reporting Dashboard And Notifications

The reporting dashboard may be generated by reporting circuit 116 ofreal-time transaction engine 110 and provide analytics obtained from theanalysis of video data uploaded from the plurality of video camera units150. The dashboard may be used to present the information in the form ofcharts, graphs, and tables. This visual display enables the user to moreeasily identify areas of concern and negative trends.

Accessibility of the dashboard may correspond with different levels ofauthentication. For example, each user may be issued a unique usernameand password. The username may correspond with a particular level ofaccess and user type, such that some reports may only be accessible to aparticular user type. For example, client device 160 may transmit ausername and profile via network 130. Authentication circuit 118 maycompare the username with a stored user profile to identify a user type.Reports, data, and other information accessible by the particular usertype may be displayed for presentation at user interface 162.

User types may correspond with an airport level, customer level, and anexecutive level. The airport level users may be provided access only toa particular stations dashboard and analytics. This level may beassigned to general managers, station managers, station safety managers,and the like. The customer level users may be provided access to onlyobservations assigned to a specific customer. The executive level usersmay be provided access to dashboards and analytics for every locationassociated with an airport facility. This level may be assigned tocorporate executive leaders and regional vice president (RVP).

A home screen may comprise access to welcome screens for each of thedifferent user levels. The home screen may provide access to threedashboards, including executive dashboard, monitoring report, andmonitoring details.

Illustrative reports are provided in FIGS. 5-15.

FIG. 5 illustrates a graphical user interface providing an executivedashboard, in accordance with some embodiment discussed herein.Executive dashboard may be accessible when an authenticated usercorresponding with an executive level accesses the system.

In some examples, the executive dashboard may provide a search tool thatcan identify safety compliance averages between a time range and/or at aparticular location (e.g., Airport A, etc.). The compliance averages maybe sorted by stages (e.g., arrival, post-arrival, post-departure,pre-arrival, pre-departure preparations, upload, etc.).

In some examples, the executive dashboard may provide a compliancepercentage in a graph, table, chart, geographical location map, or othergraphical representation.

In some examples, the safety compliance issues that were included withthe safety compliance averages. The issues may be limited in accordancewith the time range and/or at a particular location in the executivedashboard. Each issue may correspond with a predetermined value (e.g.,illustrated as “count”). The predetermined value may be higher for morecritical safety issues and lower for less critical safety issues.

FIG. 6 illustrates a graphical user interface providing a monitoringreport dashboard, in accordance with some embodiment discussed herein.In some examples, the executive dashboard may provide a search tool thatcan identify safety compliance averages between a time range, aparticular location, and/or at one or more airport gates (e.g., AirportA, Terminal 2, Gates 100, 101, 102, and 103, etc.).

FIG. 7 illustrates a graphical user interface providing a monitoringdetails dashboard, in accordance with some embodiment discussed herein.In some examples, the executive dashboard may provide a search tool thatcan identify safety compliance averages between a time range, aparticular location, one or more airport gates, and other filtersavailable by the system, including turns, categories, answers, and thelike.

FIG. 8 illustrates a graphical user interface for providing a searchfunction within an executive dashboard, in accordance with someembodiment discussed herein. In some examples, access to this GUI islimited to the executive dashboard.

In this illustrative example, a select date tool, a selected airporttool, and a select gate tool are shown. The “Select Date” window allowsfor filtering the data via a selected date range (the default is themost recent 7-day period ending with the present day. The “SelectAirports” window shows only the local airport for users with AirportAccess Level. Executive Level users are able to choose any airport wherereal-time transaction engine 110 and/or video camera unit 150 isdeployed. The “Select Gates” window allows for filtering the datarelative to gate location.

FIG. 9 illustrates a graphical user interface for providing complianceaverages, in accordance with some embodiment discussed herein. Thispanel may show the average compliance rate per “stage of operation”observed for the selected date range. Access to this GUI may not belimited to the executive level authentication as illustrated in FIG. 5.

FIG. 10 illustrates a graphical user interface for providing a totalaverage errors per turn panel, in accordance with some embodimentdiscussed herein. For example, a user operating client device 160 mayaccess this dashboard panel. The user may hover a mouse cursor over adot representing a particular airport to display an average errors perturn. In some examples, access to this GUI may not be limited to theexecutive level authentication as illustrated in FIG. 5. Other methodsof presenting the average errors per turn may be provided withoutdiverting from the scope of the disclosure.

FIG. 11 illustrates a graphical user interface for providing acompliance percentage panel, in accordance with some embodimentdiscussed herein. This panel shows a graphical illustration of thecompliance percentage each day in the selected range. For example, auser operating client device 160 may access this dashboard panel. Theuser may hover a mouse cursor over any data point to display the actualvalue. In some examples, access to this GUI may not be limited to theexecutive level authentication as illustrated in FIG. 5. Other methodsof presenting the average errors per turn may be provided withoutdiverting from the scope of the disclosure.

FIG. 12 illustrates a graphical user interface for providing a totalerror panel, in accordance with some embodiment discussed herein. Thispanel provides a total error count for each violation observed for theselected date range. In some examples, access to this GUI may not belimited to the executive level authentication as illustrated in FIG. 5.

FIGS. 13-15 may correspond with illustrative examples of monitoringreporting dashboard panels.

FIG. 13 illustrates a graphical user interface for providing a searchfilters panel, in accordance with some embodiment discussed herein. Inthis illustrative example, a select date tool, a selected airport tool,and a select gate tool are shown. The “Select Date” window allows forfiltering the data via a selected date range (the default is the mostrecent 7-day period ending with the present day. The “Select Airports”window shows only the local airport for users with Airport Access Level.Executive Level users are able to choose any airport where real-timetransaction engine 110 and/or video camera unit 150 is deployed. The“Select Gates” window allows for filtering the data relative to gatelocation.

FIG. 14 illustrates a graphical user interface for providing amonitoring details panel, in accordance with some embodiment discussedherein. The “Select Date” window allows for filtering the data via aselected date range (the default is the most recent 7-day period endingwith the present day. The “Select Airports” window shows only the localairport for users with Airport Access Level. Executive Level users areable to choose any airport where real-time transaction engine 110 and/orvideo camera unit 150 is deployed. The “Select Gates” window allows forfiltering the data relative to gate location. The “Select Turns” windowallows for filtering based on which turn of the day; 1st, 2nd, 3rd, etc.The “Select Categories” window allows for filtering based on operationalstages.

FIG. 15 illustrates a graphical user interface for providing amonitoring report, in accordance with some embodiment discussed herein.This report may be populated according to the filtering selections madein the Search Filters panel. In some examples, a checkmark appears inthe Camera column when a photo is available by clicking on the icon inthe Picture column.

FIG. 15A illustrates a graphical user interface for providing anobservation improvement report, in accordance with some embodimentdiscussed herein. This report may compare an operating phase orindividual observations compliance score to another to illustrate thedifferences between the two. The report may be limited to a selecteddate range. In some examples, the report may show the difference inimprovement or non-improvement between a date range in the effort todrive observation and/or safety issue improvement.

FIG. 15B illustrates a graphical user interface for providing areal-time alert dashboard, in accordance with some embodiment discussedherein. This report may include a set of real-time alerts by phase toindicate where an observation is being observed. The report may includeinformation in real-time or offline. The dashboard may providefunctionality to allow a user to interact with the dashboard and drilldown capabilities to determine (e.g., based on the level ofauthentication access) which airport, customer, and gate the safetyissue and/or alert is originating from.

Where components, logical circuits, or engines of the technology areimplemented in whole or in part using software, in one embodiment, thesesoftware elements can be implemented to operate with a computing orlogical circuit capable of carrying out the functionality described withrespect thereto. One such example logical circuit is shown in FIG. 16.Various embodiments are described in terms of this example logicalcircuit 1600. After reading this description, it will become apparent toa person skilled in the relevant art how to implement the technologyusing other logical circuits or architectures.

Referring now to FIG. 16, computing system 1600 may represent, forexample, computing or processing capabilities found within desktop,laptop, and notebook computers; hand-held computing devices (PDA's,smart phones, cell phones, palmtops, etc.); mainframes, supercomputers,workstations, or servers; or any other type of special-purpose orgeneral-purpose computing devices as may be desirable or appropriate fora given application or environment. Logical circuit 1600 might alsorepresent computing capabilities embedded within or otherwise availableto a given device. For example, a logical circuit might be found inother electronic devices such as, for example, digital cameras,navigation systems, cellular telephones, portable computing devices,modems, routers, WAPs, terminals and other electronic devices that mightinclude some form of processing capability.

Computing system 1600 might include, for example, one or moreprocessors, controllers, control engines, or other processing devices,such as a processor 1604. Processor 1604 might be implemented using ageneral-purpose or special-purpose processing engine such as, forexample, a microprocessor, controller, or other control logic. In theillustrated example, processor 1604 is connected to a bus 1602, althoughany communication medium can be used to facilitate interaction withother components of logical circuit 1600 or to communicate externally.

Computing system 1600 might also include one or more memory engines,simply referred to herein as main memory 1608. For example, preferablyrandom-access memory (RAM) or other dynamic memory, might be used forstoring information and instructions to be executed by processor 1604.Main memory 1608 might also be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 1604. Logical circuit 1600 might likewise includea read only memory (“ROM”) or other static storage device coupled to bus1602 for storing static information and instructions for processor 1604.

The computing system 1600 might also include one or more various formsof information storage mechanism 1610, which might include, for example,a media drive 1612 and a storage unit interface 1620. The media drive1612 might include a drive or other mechanism to support fixed orremovable storage media 1614. For example, a hard disk drive, a floppydisk drive, a magnetic tape drive, an optical disk drive, a CD or DVDdrive (R or RW), or other removable or fixed media drive might beprovided. Accordingly, storage media 1614 might include, for example, ahard disk, a floppy disk, magnetic tape, cartridge, optical disk, a CDor DVD, or other fixed or removable medium that is read by, written to,or accessed by media drive 1612. As these examples illustrate, thestorage media 1614 can include a computer usable storage medium havingstored therein computer software or data.

In alternative embodiments, information storage mechanism 1640 mightinclude other similar instrumentalities for allowing computer programsor other instructions or data to be loaded into logical circuit 1600.Such instrumentalities might include, for example, a fixed or removablestorage unit 1622 and an interface 1620. Examples of such storage units1622 and interfaces 1620 can include a program cartridge and cartridgeinterface, a removable memory (for example, a flash memory or otherremovable memory engine) and memory slot, a PCMCIA slot and card, andother fixed or removable storage units 1622 and interfaces 1620 thatallow software and data to be transferred from the storage unit 1622 tological circuit 1600.

Logical circuit 1600 might also include a communications interface 1624.Communications interface 1624 might be used to allow software and datato be transferred between logical circuit 1600 and external devices.Examples of communications interface 1624 might include a modem or softmodem, a network interface (such as an Ethernet, network interface card,WiMedia, IEEE 802.XX or other interface), a communications port (such asfor example, a USB port, IR port, RS232 port Bluetooth® interface, orother port), or other communications interface. Software and datatransferred via communications interface 1624 might typically be carriedon signals, which can be electronic, electromagnetic (which includesoptical) or other signals capable of being exchanged by a givencommunications interface 1624. These signals might be provided tocommunications interface 1624 via a channel 1628. This channel 1628might carry signals and might be implemented using a wired or wirelesscommunication medium. Some examples of a channel might include a phoneline, a cellular link, an RF link, an optical link, a network interface,a local or wide area network, and other wired or wireless communicationschannels.

In this document, the terms “computer program medium” and “computerusable medium” are used to generally refer to media such as, forexample, memory 1608, storage unit 1620, media 1614, and channel 1628.These and other various forms of computer program media or computerusable media may be involved in carrying one or more sequences of one ormore instructions to a processing device for execution. Suchinstructions embodied on the medium, are generally referred to as“computer program code” or a “computer program product” (which may begrouped in the form of computer programs or other groupings). Whenexecuted, such instructions might enable the logical circuit 1600 toperform features or functions of the disclosed technology as discussedherein.

Although FIG. 16 depicts a computer network, it is understood that thedisclosure is not limited to operation with a computer network, butrather, the disclosure may be practiced in any suitable electronicdevice. Accordingly, the computer network depicted in FIG. 16 is forillustrative purposes only and thus is not meant to limit the disclosurein any respect.

While various embodiments of the disclosed technology have beendescribed above, it should be understood that they have been presentedby way of example only, and not of limitation. Likewise, the variousdiagrams may depict an example architectural or other configuration forthe disclosed technology, which is done to aid in understanding thefeatures and functionality that can be included in the disclosedtechnology. The disclosed technology is not restricted to theillustrated example architectures or configurations, but the desiredfeatures can be implemented using a variety of alternative architecturesand configurations. Indeed, it will be apparent to one of skill in theart how alternative functional, logical, or physical partitioning andconfigurations can be implemented to implement the desired features ofthe technology disclosed herein. Also, a multitude of differentconstituent engine names other than those depicted herein can be appliedto the various partitions.

Additionally, with regard to flow diagrams, operational descriptions andmethod claims, the order in which the steps are presented herein shallnot mandate that various embodiments be implemented to perform therecited functionality in the same order unless the context dictatesotherwise.

Although the disclosed technology is described above in terms of variousexemplary embodiments and implementations, it should be understood thatthe various features, aspects and functionality described in one or moreof the individual embodiments are not limited in their applicability tothe particular embodiment with which they are described, but instead canbe applied, alone or in various combinations, to one or more of theother embodiments of the disclosed technology, whether or not suchembodiments are described and whether or not such features are presentedas being a part of a described embodiment. Thus, the breadth and scopeof the technology disclosed herein should not be limited by any of theabove-described exemplary embodiments.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like; the term“example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof; the terms “a” or“an” should be read as meaning “at least one,” “one or more” or thelike; and adjectives such as “conventional,” “traditional,” “normal,”“standard,” “known” and terms of similar meaning should not be construedas limiting the item described to a given time period or to an itemavailable as of a given time, but instead should be read to encompassconventional, traditional, normal, or standard technologies that may beavailable or known now or at any time in the future. Likewise, wherethis document refers to technologies that would be apparent or known toone of ordinary skill in the art, such technologies encompass thoseapparent or known to the skilled artisan now or at any time in thefuture.

The presence of broadening words and phrases such as “one or more,” “atleast,” “but not limited to” or other like phrases in some instancesshall not be read to mean that the narrower case is intended or requiredin instances where such broadening phrases may be absent. The use of theterm “engine” does not imply that the components or functionalitydescribed or claimed as part of the engine are all configured in acommon package. Indeed, any or all of the various components of anengine, whether control logic or other components, can be combined in asingle package or separately maintained and can further be distributedin multiple groupings or packages or across multiple locations.

Additionally, the various embodiments set forth herein are described interms of exemplary block diagrams, flow charts and other illustrations.As will become apparent to one of ordinary skill in the art afterreading this document, the illustrated embodiments and their variousalternatives can be implemented without confinement to the illustratedexamples. For example, block diagrams and their accompanying descriptionshould not be construed as mandating a particular architecture orconfiguration.

What is claimed is:
 1. A video analytics system for real-time monitoringand assessment of airplane ramp safety processes, the video analyticssystem comprising: a computer memory storing rules relating to theairplane ramp safety processes; at least two portable, video capturedevices located at different predetermined positions with respect to anaircraft at an airport ramp, wherein the video capture devices include awireless transmitter and are configured to: capture video and datarelated to movements and positioning of objects and services beingprovided relative to the aircraft at the airport ramp; and transmitcaptured video footage and data via a wireless network to a server; anda server comprising a processor, programmed with computer instructions,which when executed cause the processor to operate a real-timetransaction engine configured to: perform video analytics on thecaptured video and data to generate video analytics data; compare thevideo analytics data to the stored rules relating to the airplane rampsafety processes, including to determine any safety conditions; storeresults of the comparison in the computer memory; and output a real-timenotification if a safety condition is determined.
 2. The video analyticssystem of claim 1, wherein the rules relating to the airplane rampsafety processes comprise a first set of rules for a first aircraft typeand a second set of rules for a second aircraft type; and thepredetermined positions of the video capture devices comprise a firstset of positions for the first aircraft type and a second set ofpositions for the second aircraft type.
 3. The video analytics system ofclaim 1, wherein the video capture devices are selectively positioned toprovide a range of visibility to cover a safety diamond area of a parkedaircraft.
 4. The video analytics system of claim 1, wherein the storedrules comprise rules relating to a set of safety procedures including anidentification of at least a subset of the following: i. proper uniformsare worn with safety vests that are identified as zipped up, ii. anappropriate placement of safety cones surrounding the aircraft, groundservice equipment and other applicable area, iii. use of wheel chocks onthe aircraft, ground service equipment and baggage carts, iv. anidentification and verification of by-pass pins/switches are in placebased on an aircraft type, v. an identification and verification ofpre-flight and post-flight safety huddles, vi. verification thatappropriate measures are deployed for any ground service equipment thatapproaches the aircraft, including the use of multiple identified brakechecks and minimum distances from the aircraft, vii. verification thatground service equipment only operates within an allotted area ofoperation, viii. an identification and verification that foreign objectdebris walks are being conducted prior to the aircraft entering theairport ramp and prior to the aircraft leaving the airport ramp, ix. anidentification and verification that wing walkers and marshals are intheir appropriate position prior to when the aircraft starts to approachthe airport ramp operation or starts its pushback out of the airportramp operation, and x. an identification and verification that safetyhand rails are being raised and used by ground service personnel as theywalk on any ground service equipment to enter or exit the aircraft. 5.The video analytics system of claim 1, wherein the server comprises aweb-based portal including a graphical user interface configured topresent results of observations, analytical reports, alertnotifications, and airport and gate configurations.
 6. The videoanalytics system of claim 5, wherein the web-based portal is configuredto display real-time alerts generated by a real-time alert managementsystem.
 7. The video analytics system of claim 1, comprising a real-timealert management system configured to notify a pre-configured set ofresources if a safety condition is identified.
 8. The video analyticssystem of claim 1, comprising a real-time alert management systemconfigured to transmit notifications of identified safety conditions viaa mobile messaging communication protocol.
 9. The video analytics systemof claim 1, comprising a real-time alert management system configured togenerate alert information if a safety condition is identified,including an airport and gate from which the alert informationoriginated.
 10. The video analytics system of claim 1, furthercomprising a set of machine learning models each associated with anaircraft type and the machine learning model for the aircraft type beingconfigured to process the stored results of the comparison of the videoanalytics data generated from the video and data related to movementsand positioning of objects and services provided relative to theaircraft to generate recommended improvements to the stored rulesrelating to the airplane ramp safety processes.
 11. The video analyticssystem of claim 1, comprising a safety conditions verification modulefor designating identified safety conditions as false positives or falsenegatives and providing feedback to the set of machine learning modelsto refine accuracy of the stored rules.
 12. The video analytics systemof claim 1, comprising a compliance score generation module configuredto generate a compliance score corresponding to a number of rules thatare satisfied based on the comparison of the video analytics data to thestored rules.
 13. A computer-implemented method for real-time monitoringand assessment of airplane ramp safety processes, thecomputer-implemented method comprising: capturing video and data, byvideo capture devices, related to movements and positioning of objectsand services being provided relative to an aircraft at an airport ramp,wherein the video capture devices are located at different predeterminedpositions with respect to the aircraft at the airport ramp, and whereinthe video capture devices include a wireless transmitter; transmittingcaptured video footage and data via a wireless network to a server;performing, by a server operating a real-time transaction engine, videoanalytics on the captured video footage and data to generate videoanalytics data; comparing the video analytics data to the stored rulesrelating to the airplane ramp safety processes, including to determineany safety conditions; storing the results of the comparison in acomputer memory; and outputting a real-time notification if a safetycondition is determined.
 14. The computer-implemented method of claim13, wherein the rules relating to the airplane ramp safety processescomprise a first set of rules for a first aircraft type and a second setof rules for a second aircraft type; and the predetermined positions ofthe video capture devices comprise a first set of positions for thefirst aircraft type and a second set of positions for the secondaircraft type.
 15. The computer-implemented method of claim 13, whereinthe video capture devices are selectively positioned to provide a rangeof visibility to cover a safety diamond area of a parked aircraft. 16.The computer-implemented method of claim 13, wherein the server furthercomprising a real-time alert management system configured to transmitnotifications of identified safety conditions via a mobile messagingcommunication protocol.
 17. The computer-implemented method of claim 13,wherein the server further comprising a real-time alert managementsystem configured to generate alert information if a safety condition isidentified, including an airport and gate from which the alertinformation originated.
 18. The computer-implemented method of claim 13,wherein the server further comprising a set of machine learning modelseach associated with an aircraft type and the machine learning model forthe aircraft type being configured to process the stored results of thecomparison the video analytics data generated from the video and datarelated to movements and positioning of objects and services providedrelative to the aircraft to generate recommended improvements to thestored rules relating to the airplane ramp safety processes.
 19. Thecomputer-implemented method of claim 13, wherein the server furthercomprising a safety conditions verification module for designatingidentified safety conditions as false positives or false negatives andproviding feedback to the set of machine learning models to refineaccuracy of the stored rules.
 20. The computer-implemented method ofclaim 13, wherein the server further comprising a compliance scoregeneration module configured to generate a compliance scorecorresponding to a number of rules that are satisfied based on thecomparison of the video analytics data to the stored rules.